FDR_TEST: A SAS Macro for Calculating New Methods of Error Control in Multiple Hypothesis Testing

The testing of multiple null hypotheses in a single study is a common occurrence in applied research. The problem of Type I error inflation or probability pyramiding in such contexts has been well-known for many years. General procedures for the control of Type I error rates in multiple testing are the Bonferroni procedure and its’ more recent modifications. These procedures partition a desired level of Familywise error across the set of hypotheses being tested. Recent work on multiple testing by Benjamini and Hochberg (1993, 1995, 2000) has focused on controlling the False Discovery Rate (FDR) rather than rates of Type I error. The adaptive (BH-A) and nonadaptive (BH) procedures for controlling the FDR in a set of tests promise increases in statistical power relative to other procedures. This paper presents a SAS macro that calculates probabilities under five decision rules that may be used in multiple testing (per hypothesis rule, Bonferroni, Hochberg, BH, BH-A) The macro evaluates a set of probabilities that are supplied as an input. Macro outputs include the results of the five decision rules applied to the set of probabilities. The paper provides a demonstration of the SAS/IML code and examples of the application of the code in simulation studies.